MASTER THESIS by Paul Fink Ensemble methods for classification trees under imprecise probabilities

In this master thesis some properties of bags of imprecise classification trees, as introduced in Abellán and Masegosa (2010), are analysed. In the beginning the statistical background of imprecise classification trees is outlined – starting with an overview on measuring uncertainty within the concept of Dempster–Shafer theory is presented, followed by a discussion of its application in a tree–growing–algorithm, which employs the so–called Imprecise Dirichlet Model in the splitting process. The motivation of so–called ensemble methods is to reduce the instability of a single classification tree, increasing its predictive accuracy, but at cost of structural interpretability. A description of the well known ensemble methods, such as bagging, random forests and boosting, is given along with two approaches, generating the ensemble by allowing more than one splitting variable within a node. In the next step a bag of imprecise classification trees is generated; following, its sensitivity in relation to different ensemble aggregation/fusion rules (majority voting, disjunction and average rule), the external stopping criterion of minimal observations within a node and the main parameter of the Imprecise Dirichlet Model is analysed by a simulation study. The results of the simulation indicate that the commonly applied majority voting rule is also a fair choice for imprecise classification ensembles. Regarding the external stopping criterion the simulation indicates that such may be neglected, while the parameter does highly affect the predictive accuracy, favouring lower values of it.